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In the general two-Higgs-doublet model, including the minimal supersymmetric standard model (MSSM) as its specific 
realization, the two heavy neutral Higgs bosons are nearly degenerate in the decoupling limit. If the theory is CP- 
noninvariant, the mixing between the heavy states can strongly be affected by the decay widths. We develop the 
formalism describing this CP— violating non-hermitian mixing and provide some interesting experimental signatures 
of the CP— violating mixing at a 77 collider with polarized beams in the context of the CP-noninvariant MSSM. 



1. INTRODUCTION 



^ I The MSSM is a specific realization of general scenarios that include two doublet fields in the Higgs sector. After 

' three Goldstone fields are absorbed by electroweak gauge bosons, the remaining five fields give rise to physical states. 

^-H . At the tree level, the MSSM Higgs sector is CP invariant, with two CP-even and one CP-odd neutral states. However, 

[ the MSSM offers new sources of CP violation, which render the Higgs sector CP-noninvariant at the loop level. In 

00 ' such CP-noninvariant theories the three neutral states mix to form a triplet with both even and odd components in 

■ the wave-functions under CP transformations 0, |3) 0] ■ The mixing can become very large if the states are nearly 
mass-degenerate. This situation is naturally realized for supersymmetric theories in the decoupling limit Q| in which 

^ two of the neutral states are heavy. 

^ ' In the present report we describe a simple quantum mechanical (QM) formalism for the CP-violating resonant 

^ ^ H/A mixing in the decoupling limit and then discuss some experimental signatures of the CP-violating mixing in 

, Higgs production and decay processes at a photon collider with polarized photon beams. 

> 

>^ 2. MIXING FORMALISM 

The self-interaction of two Higgs doublets in a CP-noninvariant theory is generally described by the potential Q 
V = m?i$l$i+m22$J$2-K2*I*2 + h.c.] + ^Ai($|$i)2 + iA2($^$2)' 

+ A3($I$l)($^$2) + A4($I$2)(*^$l) + |^A5($I$2)' + [A6($l$l) + A7($|$2)] *I$2 + h.C.j (1) 

where $1,2 denote two complex Y — 1, SU(2)i iso-doublet scalar fields. The coefficients are in general all non-zero. 
The parameters mi2, X^^j can be complex, incorporating the CP-noninvariant elements in the interactions. The 
neutral components of the scalar fields <&! and $2 are assumed to develop non-zero vacuum expectation values 
(vevs) (0?) = V1/V2 and (0^) = V2/V2, which can be chosen real and positive without loss of generality. As usual, 
V = {vj+vD^^^ = 246 GeV. 

It is useful to rotate the Higgs fields $1^2 to the $a,b basis with the angle /3 satisfying tan/3 = V2/V1 (we exploit 
the abbreviations tp — tan/3, cp = cos/3, S2/3 = sin 2/3 etc.). In this basis only the field $a develops a non-zero vev 



*'^= 1 , ..^ (2) 
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and the three fields G*'" can be identified as the would-be Goldstone bosons, while H^^Haj, and A give rise to 
physical Higgs bosons. The real mass matrix A^g of neutral Higgs fields in the basis of i/^, Hi,, A, which is hermitian 
and symmetric by CPT invariance, can easily be derived from the Higgs potential after the rotations: 

A A Ap 

Mln ^v^\ -X X-Xa + Ml/v^ -Ap | (3) 
-Ap -Ap Ml/v^ 

after eliminating "^12 from the minimization conditions, and exchanging m^|^ for the auxiliary parameter M^. 

It is defined by the relation 

rnl^ = \{Mls2p + v^{Xi-s,^j + X^cl + X^sD) (4) 

and it will be one of the key parameters in the system. The A, A and Xa parameters are functions of the real parts, 
while Ap and Ap are functions of the imaginary parts of the parameters A^ in Eq. their explicit form can be found 
in Ref. % 

In a CP-invariant theory all Xi couplings are real and the off-diagonal elements Ap,Ap vanish. Thus the neutral 
mass matrix breaks into the CP-even 2x2 part, and the [stand-alone] CP-odd part. The 2x2 part gives rise to 
two CP-even neutral mass eigenstates h and _ff , while Ma is identified as the mass of the CP-odd Higgs boson A. In 
the CP-violating case, however, all three states mix leading to ffi,2,3 mass eigenstates with no definite CP parities. 

For small mass differences, the mixing of the states is strongly affected by their widths. This is a well-known 
phenomenon for resonant mixing and has also been recognized for the Higgs sector 7]. The hermitian mass 
matrix (PJ has therefore to be supplemented by the anti-hermitian part —iMT incorporating the decay matrix jsj 

^Ml- iMT (5) 

This matrix includes the widths of the Higgs states in the diagonal elements as well as the transition elements within 
any combination of pairs. They are particularly important in the case of nearly mass-degenerate states. All these 
elements (A/r)^^'^ are built up by loops of the fields (AB) in the self-energy matrix (hahb) of the Higgs fields. 

In general, the light Higgs boson, the fermions and electroweak gauge bosons, and in supersymmetric theories, 
gauginos, higgsinos and scalar states may contribute to the loops in the propagator matrix. In the physically 
interesting case of decoupling, the mixing structure simplifies considerably allowing for a very simple and transparent 
approach ^] . Alternative approach requires a full coupled-channel analysis |^ . 

2.1. Decoupling Limit 

The decoupling limit 4J is defined by the inequality AI^ ^ |Ai| with |Ai| ^ 0(1). In this limit the Ha state 
becomes the CP-even light Higgs boson h and decouples from Hi, and A. The heavy states H = Hi, and A are nearly 
mass degenerate, which turns out to be crucial for large mixing effects between H and A. It is therefore enough to 
consider a lower-right 2x2 submatrix of the matrix |j2Jl for the heavy H/A states which we write as follows 



■^^^ - ' ^Ia Ml - ^MATA ' 



where Aj^^ also consists of a real dissipative part and an imaginary absorptive part. Moreover, the couplings 
of the heavy Higgs bosons to gauge bosons and their supersymmetric partners are suppressed. In the case of all 
supersymmetric particle contributions to be suppressed either by couplings or by phase space in MP, it is sufficient 
to consider only loops built up by the light Higgs boson and top quark- for the explicit form of the light Higgs boson 
and top quark loop contributions to the matrix MP, we refer to Ref. The loops also contribute to the real part 
of the mass matrix, either renormalizing the A parameters of the Higgs potential or generating such parameters if 
not present yet at the tree level. 
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2.2. Physical Masses and States 

The symmetric complex mass-squared matrix A^'^ in Eq.(|SJl can be diagonalized through a complex rotation 

MIh ' '^W"^ \= CMl^C-' (7) 

where the mixing matrix and the mixing angle are given by 

r-( cos6i sin 61 \ Y-it«n9fl- 

1^ - sin cos j ' ^ - 2 t^i^ - ~Ml~2 [MhTh - M^Ta] ^ ' 

A non-vanishing (complex) mixing parameter X 7^ requires CP-violating transitions between H and A either in 
the real mass matrix, Xp 7^ 0, or in the decay mass matrix, (MT)ha 7^ 0, [or both]. However, note that even for 
nearly degenerate masses, the mixing could be suppressed if the widths were significantly different. As a result, the 
mixing phenomena are strongly affected by the form of the decay matrix MT. Since the difference of the widths 
enters through the denominator in X, the modulus \X\ becomes large for small differences and small widths. 

The mixing shifts the Higgs masses and widths in a characteristic pattern J3J . The two complex mass values after 
and before diagonalization are related by the complex mixing angle 9: 



M^3 - - ^ {Mh.Ths - Mh.Th,) = [M^ - Mjj - i{MAT - MhTh)] x + 4^2 (9) 

Since the eigenstates of the complex, non-hermitian matrix are no longer orthogonal, the ket and bra mass 
eigenstates have to be defined separately: \Hi) — Cia\Ha) and {Hi\ — Cia{Ha\ {i = 2,3 and Ha = H,A). The final 
state F in heavy Higgs formation from the initial state / is, then, described with the amplitude 

(FIHII) = V (F\H,) ^ (HAI) (10) 

where the sum runs only over diagonal transitions in the mass-eigenstate basis. 



3. EXPERIMENTAL SIGNATURES 

To illustrate the general QM results in a realistic example, we adopt a specific MSSM scenario with the source 
of CP-violation localized in the complex trilinear coupling At of the soft supersymmetry breaking part involving 
the top squark.^ All other interactions are assumed to be CP-conserving. For (j)A ^ 0, tt, the stop-loop corrections 
induce the CP-violation in the effective Higgs potential (Q. The effective Xi parameters have been calculated in 
Ref. Q to two-loop accuracy; to illustrate the crucial points we take the dominant one-loop t/i contributions. 

More specifically, we take a typical set of parameters from Ref. ^3 j 

Ms = 0.5 TeV, = 1.0 TeV, 1.0 TeV; tan/? = 5 (11) 

and change the phase (pA of the trilinear parameter At . With (j)A = wc find the following values of the light and 
heavy Higgs masses and decay widths, and the stop masses: 

Mft = 129.6 GeV, M// = 500.3 GeV, A/a = 500.0 GeV; = 1.2 GeV, = 1.5 GeV; mz = 372/647 GeV (12) 

1/2 

Clearly, with the mass splitting of 0.3 GeV, the heavy Higgs states are not distinguishable. When the phase (pA is 



'^This assignment is compatible with the bounds on CP-violating SUSY phases from experiments on electric dipole moments llOl . 
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(a) Mixing Parameter (b) Mass Shift (c) Width Shift 




Figure 1: The <^a dependence of (a) the mixing parameter X and of the shifts of (b) masses and (c) widths with the phase 
(j)A evolving from to tt for tan/3 = 5, Ma = 0.5 TeV and couphngs as specified in the text; in (b,c) the mass and width 
differences without mixing are shown by the broken hnes. 5Re/9mX(27r — 0a) = +JRe/— 9mX((;/)A) for angles above vr. 



turned on,^ the CP composition, the masses and the decay widths of heavy states are strongly affected, as shown 
in Figs.nja), (b) and (c), while the mass of the light Higgs boson h is not. The heavy two-state system shows a 
very sharp resonant CP-violating mixing, purely imaginary a little above (jjA = 37r/4, Fig. ^a). The mass shift is 
enhanced by more than an order of magnitude if the CP-violating phase rises to non-zero values, reaching a maximal 
value of ~ 5.3 GeV; the width shift changes between —0.3 and +0.4 GeV. As a result, the two mass-eigenstates 
should become clearly distinguishable at future colliders, in particular at a photon collider Moreover, both 

states have significant admixtures of CP-even and CP-odd components in the wave-functions. Since 77 colliders 
offer unique conditions for probing the CP-mixing |l3l Il4l [isj. we discuss two experimental examples: (a) Higgs 
formation in polarized 77 collisions and (b) polarization of top quarks in Higgs decays, where spectacular signatures 
of resonant mixing can be expected. 

(a) The amplitude of the reaction 77 ^ Hi F is a. superposition of H2 and H3 exchanges. For equal helicities 
A = ±1 of the two photons, the amplitude reads 

-Mf - E (^1^^) s-Mll^M,T,^ ISlis) + ^^P^ (s)] (13) 

where y/s is the 77 energy and the loop-induced 77i/i scalar and pseudoscalar form factors, >5'7(s) and P^is), are 
related to the well-known conventional jjH/A form factors, S]^ and P^ j^- For their relation and explicit form 
we refer to Refs. and 0|. In our scenario the Higgs-tt couplings are assumed to be CP-conserving, implying 
negligible top-loop contributions to and SJ^ since the gluino mass is sufficiently heavy compared with the stop 
masses, while the ti loop generates a non-negligible CP-violating amplitude Sj^. In the region of strong mixing on 
which we focus, however, the CP-violating vertex corrections have only a small effect on the experimental asymmetries 
compared with the large impact of CP-violating Higgs-boson mixing. 

Polarized photons provide a very powerful tool to investigate the CP properties of Higgs bosons. With linearly 
polarized photons one can project out the CP-even and CP-odd components of the Hi wave-functions by arranging 
the photon polarization vectors to be parallel or perpendicular. On the other hand, circular polarization provides us 
with a direct insight into the CP- violating nature of Higgs bosons. Two asymmetries are of interest 

Aun = Ahel = '^^^ 7 (14) 
(T\\ + (7j_ 0-++ + 

where (T|| , a±_ and it_|_+, a are the corresponding total 77 fusion cross sections for linear and circular polarizations, 

respectively. Though CP-even, the asymmetry Aun can serve as a powerful tool nevertheless to probe CP-violating 



^With one phase ijiAi the complex mixing parameter X obeys the relation X{2-k — <f>A) = ^*(</'a)i implying all CP-even quantities 
symmetric and all CP-odd quantities anti— symmetric about tt. 
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admixtures to the Higgs states since |.4/i„| < 1 requires both and non-zero couphngs. A more direct probe of 
CP-violation due to H/A mixing is provided by the CP-odd (and also CPT-odd) asymmetry Aim- 



.1, J J..J,J J.JC 
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Figure 2: The 4)a dependence of the CP-even and CP-odd correlators, Ann (left panel) and Ahei (center panel), at the poles 
of H2 and H3, respectively, and the 77 energy dependence (right panel) of the correlators, Aiin,hei, for (j>A ~ 3n /A in the 
production process 77 Hi in the limit in which H/A mixing is the dominant CP-violating effect. The same parameter set 
as in Fig.Qis employed. The vertical lines on the right panel mark positions of the two mass eigenvalues, Adn^ and Mh^- 

Fig-Hlshow the (j)A dependence of the asymmetries Aim and Ahei at the poles of H2 and of iJs, respectively, for the 
same parameter set as in Fig. Q] and with the common SUSY scale Mq^ = Mf^ = A/5 = 0.5 TeV for the soft SUSY 
breaking top squark mass parameters. By varying the 77 energy from below Mh^ to above AIh2 , the asymmetries, 
Aiin (blue solid line) and Ahei (red dashed line), vary from —0.39 to 0.34 and from —0.29 to 0.59, respectively, as 
demonstrated on the right panel of Fig. |2] with (pA — 37r/4, a phase value close to resonant CP-mixing. 

(b) A second observable of interest is the polarization of the top quarks in Hi decays produced by 77 fusion or 
elsewhere in various production processes at an e+e~ linear collider and LHC i/2,3 tt- Even if the H/Att couplings 
are [approximately] CP-conserving, the complex rotation matrix C may mix the CP-even H and CP-odd A states 
leading to CP-violation. In the production-decay process -f-f ^ Hi ^ tt, two CP-even and CP-odd correlators 
between the transverse t and t polarization vectors s± , s± 

C|| = (sj_ • s_l) and, C± = (pt ■ (s_l x s±)) (15) 

can be extracted from the azimuthal-angle correlation between the two decay planes t bW~^ and t —> bW~ . 

Fig. 13 shows the (t>A dependence of the CP-even and CP-odd asymmetries, Cy and C±, at the poles of H2 and 
of H3, left and center panels respectively. If the invariant tt energy is varied throughout the resonance region, the 
correlators Cy (blue solid line) and C± (red dashed line) vary characteristically from —0.43 to —0.27 [non-uniformly] 
and from 0.84 to —0.94, respectively, as shown in the right panel of Fig.|31 



L J ^ I J I J L I I I 



i^^^^j^ -ii.'^ ' ' dit ' 0.6" Q.8 '-' "TF^0" ^-^q:2 ^:^ ' ' ' 6. ' 6 0.ij ' ' 




500 
E„ [GeV] 



Figure 3: The (j)A dependence of the CP-even and CP-odd correlators, C\\ (left panel) and C± (center panel), at the pole of H2 
and Hs and the invariant tt energy dependence (right panel) of the correlators C||_x for (t>A ~ Sn/i in the production-decay 
chain 'yy —* Hi ^ tt. [Same SUSY parameter set as in Fig|5|] 
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4. CONCLUSIONS 

Exciting mixing effects can occur in the supersymmetric Higgs sector if CP-noninvariant interactions are present. 
In the decoupHng regime these effects can become very large, leading to interesting experimental consequences. Higgs 
formation in 77 collisions with polarized beams proves particularly interesting for observing such effects. However, 
exciting experimental effects are also predicted in such scenarios for ti final-state analyses in decays of the heavy 
Higgs bosons at LHC and in the e+e" mode of linear colliders. 

Detailed experimental simulations would be needed to estimate the accuracy with which the asymmetries presented 
here can be measured. Though not easy to measure, the large magnitude and the rapid, significant variation of the 
CP-even and CP-odd asymmetries through the resonance region with respect to both the phase 4>a and the 77 
energy would be a very interesting effect to observe in any case. 
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